{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(basic-demo)=\n",
    "# Feature store example (stocks)\n",
    "\n",
    "This notebook demonstrates the following:\n",
    "- Generate features and feature-sets\n",
    "- Build complex transformations and ingest to offline and real-time data stores\n",
    "- Fetch feature vectors for training\n",
    "- Save feature vectors for re-use in real-time pipelines\n",
    "- Access features and their statistics in real-time\n",
    "\n",
    "```{admonition} Note\n",
    "By default, this demo works with the online feature store, which is currently not part of the Open Source MLRun default deployment.\n",
    "```\n",
    "\n",
    "**In this section**\n",
    "- [Get started](#get-started)\n",
    "- [Create sample data for demo](#create-sample-data-for-demo)\n",
    "- [Define, infer and ingest feature sets](#define-infer-and-ingest-feature-sets)\n",
    "- [Get an offline feature vector for training](#get-an-offline-feature-vector-for-training)\n",
    "- [Initialize an online feature service and use it for real-time inference](#initialize-an-online-feature-service-and-use-it-for-real-time-inference)\n",
    "\n",
    "## Get started\n",
    "\n",
    "Install the latest MLRun package and restart the notebook.\n",
    "\n",
    "Setting up the environment and project:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2023-02-05 11:43:17,605 [info] Created and saved project stocks: {'from_template': None, 'overwrite': False, 'context': './', 'save': True}\n",
      "> 2023-02-05 11:43:17,607 [info] created project stocks and saved in MLRun DB\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<mlrun.projects.project.MlrunProject at 0x7f689811ea10>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import mlrun\n",
    "\n",
    "mlrun.get_or_create_project(\"stocks\", \"./\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create sample data for demo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": [
     "hide-cell"
    ]
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "quotes = pd.DataFrame(\n",
    "    {\n",
    "        \"time\": [\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.023\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.023\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.030\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.041\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.048\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.049\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.072\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.075\"),\n",
    "        ],\n",
    "        \"ticker\": [\"GOOG\", \"MSFT\", \"MSFT\", \"MSFT\", \"GOOG\", \"AAPL\", \"GOOG\", \"MSFT\"],\n",
    "        \"bid\": [720.50, 51.95, 51.97, 51.99, 720.50, 97.99, 720.50, 52.01],\n",
    "        \"ask\": [720.93, 51.96, 51.98, 52.00, 720.93, 98.01, 720.88, 52.03],\n",
    "    }\n",
    ")\n",
    "\n",
    "trades = pd.DataFrame(\n",
    "    {\n",
    "        \"time\": [\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.023\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.038\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.048\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.048\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.048\"),\n",
    "        ],\n",
    "        \"ticker\": [\"MSFT\", \"MSFT\", \"GOOG\", \"GOOG\", \"AAPL\"],\n",
    "        \"price\": [51.95, 51.95, 720.77, 720.92, 98.0],\n",
    "        \"quantity\": [75, 155, 100, 100, 100],\n",
    "    }\n",
    ")\n",
    "\n",
    "\n",
    "stocks = pd.DataFrame(\n",
    "    {\n",
    "        \"ticker\": [\"MSFT\", \"GOOG\", \"AAPL\"],\n",
    "        \"name\": [\"Microsoft Corporation\", \"Alphabet Inc\", \"Apple Inc\"],\n",
    "        \"exchange\": [\"NASDAQ\", \"NASDAQ\", \"NASDAQ\"],\n",
    "    }\n",
    ")\n",
    "\n",
    "import datetime\n",
    "\n",
    "\n",
    "def move_date(df, col):\n",
    "    max_date = df[col].max()\n",
    "    now_date = datetime.datetime.now()\n",
    "    delta = now_date - max_date\n",
    "    df[col] = df[col] + delta\n",
    "    return df\n",
    "\n",
    "\n",
    "quotes = move_date(quotes, \"time\")\n",
    "trades = move_date(trades, \"time\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### View the demo data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "tags": [
     "hide-output"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>ticker</th>\n",
       "      <th>bid</th>\n",
       "      <th>ask</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2021-05-23 09:04:07.013574</td>\n",
       "      <td>GOOG</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.93</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2021-05-23 09:04:07.013574</td>\n",
       "      <td>MSFT</td>\n",
       "      <td>51.95</td>\n",
       "      <td>51.96</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2021-05-23 09:04:07.020574</td>\n",
       "      <td>MSFT</td>\n",
       "      <td>51.97</td>\n",
       "      <td>51.98</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2021-05-23 09:04:07.031574</td>\n",
       "      <td>MSFT</td>\n",
       "      <td>51.99</td>\n",
       "      <td>52.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2021-05-23 09:04:07.038574</td>\n",
       "      <td>GOOG</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.93</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2021-05-23 09:04:07.039574</td>\n",
       "      <td>AAPL</td>\n",
       "      <td>97.99</td>\n",
       "      <td>98.01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2021-05-23 09:04:07.062574</td>\n",
       "      <td>GOOG</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.88</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2021-05-23 09:04:07.065574</td>\n",
       "      <td>MSFT</td>\n",
       "      <td>52.01</td>\n",
       "      <td>52.03</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                        time ticker     bid     ask\n",
       "0 2021-05-23 09:04:07.013574   GOOG  720.50  720.93\n",
       "1 2021-05-23 09:04:07.013574   MSFT   51.95   51.96\n",
       "2 2021-05-23 09:04:07.020574   MSFT   51.97   51.98\n",
       "3 2021-05-23 09:04:07.031574   MSFT   51.99   52.00\n",
       "4 2021-05-23 09:04:07.038574   GOOG  720.50  720.93\n",
       "5 2021-05-23 09:04:07.039574   AAPL   97.99   98.01\n",
       "6 2021-05-23 09:04:07.062574   GOOG  720.50  720.88\n",
       "7 2021-05-23 09:04:07.065574   MSFT   52.01   52.03"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "quotes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "tags": [
     "hide-output"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>ticker</th>\n",
       "      <th>price</th>\n",
       "      <th>quantity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2021-05-23 09:04:07.041766</td>\n",
       "      <td>MSFT</td>\n",
       "      <td>51.95</td>\n",
       "      <td>75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2021-05-23 09:04:07.056766</td>\n",
       "      <td>MSFT</td>\n",
       "      <td>51.95</td>\n",
       "      <td>155</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2021-05-23 09:04:07.066766</td>\n",
       "      <td>GOOG</td>\n",
       "      <td>720.77</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2021-05-23 09:04:07.066766</td>\n",
       "      <td>GOOG</td>\n",
       "      <td>720.92</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2021-05-23 09:04:07.066766</td>\n",
       "      <td>AAPL</td>\n",
       "      <td>98.00</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                        time ticker   price  quantity\n",
       "0 2021-05-23 09:04:07.041766   MSFT   51.95        75\n",
       "1 2021-05-23 09:04:07.056766   MSFT   51.95       155\n",
       "2 2021-05-23 09:04:07.066766   GOOG  720.77       100\n",
       "3 2021-05-23 09:04:07.066766   GOOG  720.92       100\n",
       "4 2021-05-23 09:04:07.066766   AAPL   98.00       100"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trades"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "tags": [
     "hide-output"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ticker</th>\n",
       "      <th>name</th>\n",
       "      <th>exchange</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MSFT</td>\n",
       "      <td>Microsoft Corporation</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>GOOG</td>\n",
       "      <td>Alphabet Inc</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>AAPL</td>\n",
       "      <td>Apple Inc</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  ticker                   name exchange\n",
       "0   MSFT  Microsoft Corporation   NASDAQ\n",
       "1   GOOG           Alphabet Inc   NASDAQ\n",
       "2   AAPL              Apple Inc   NASDAQ"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stocks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define, infer and ingest feature sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mlrun.feature_store as fstore\n",
    "from mlrun.feature_store.steps import *\n",
    "from mlrun.features import MinMaxValidator"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build and ingest simple feature set (stocks)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>exchange</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ticker</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>MSFT</th>\n",
       "      <td>Microsoft Corporation</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GOOG</th>\n",
       "      <td>Alphabet Inc</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AAPL</th>\n",
       "      <td>Apple Inc</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         name exchange\n",
       "ticker                                \n",
       "MSFT    Microsoft Corporation   NASDAQ\n",
       "GOOG             Alphabet Inc   NASDAQ\n",
       "AAPL                Apple Inc   NASDAQ"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# add feature set without time column (stock ticker metadata)\n",
    "stocks_set = fstore.FeatureSet(\"stocks\", entities=[fstore.Entity(\"ticker\")])\n",
    "fstore.ingest(stocks_set, stocks, infer_options=fstore.InferOptions.default())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build an advanced feature set - with feature engineering pipeline\n",
    "Define a feature set with custom data processing and time aggregation functions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a new feature set\n",
    "quotes_set = fstore.FeatureSet(\"stock-quotes\", entities=[fstore.Entity(\"ticker\")])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Define a custom pipeline step (python class)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MyMap(MapClass):\n",
    "    def __init__(self, multiplier=1, **kwargs):\n",
    "        super().__init__(**kwargs)\n",
    "        self._multiplier = multiplier\n",
    "\n",
    "    def do(self, event):\n",
    "        event[\"multi\"] = event[\"bid\"] * self._multiplier\n",
    "        return event"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Build and show the transformation pipeline**\n",
    "\n",
    "Use `storey` stream processing classes along with library and custom classes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n",
       " -->\n",
       "<!-- Title: mlrun&#45;flow Pages: 1 -->\n",
       "<svg width=\"1012pt\" height=\"98pt\"\n",
       " viewBox=\"0.00 0.00 1012.34 98.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 94)\">\n",
       "<title>mlrun&#45;flow</title>\n",
       "<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-94 1008.3418,-94 1008.3418,4 -4,4\"/>\n",
       "<!-- _start -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>_start</title>\n",
       "<polygon fill=\"#d3d3d3\" stroke=\"#000000\" points=\"38.5476,-27.0493 40.698,-27.1479 42.8263,-27.2953 44.9236,-27.4913 46.9815,-27.7353 48.9917,-28.0266 50.9463,-28.3645 52.8377,-28.7479 54.6587,-29.1759 56.4025,-29.6472 58.0628,-30.1606 59.634,-30.7147 61.1107,-31.308 62.4882,-31.9388 63.7625,-32.6054 64.9302,-33.3059 65.9882,-34.0385 66.9343,-34.8012 67.7669,-35.5918 68.4849,-36.4082 69.0878,-37.2481 69.5758,-38.1093 69.9496,-38.9894 70.2102,-39.886 70.3595,-40.7965 70.3997,-41.7186 70.3334,-42.6497 70.1636,-43.5873 69.8937,-44.5287 69.5276,-45.4713 69.0691,-46.4127 68.5225,-47.3503 67.8923,-48.2814 67.1831,-49.2035 66.3996,-50.114 65.5464,-51.0106 64.6285,-51.8907 63.6504,-52.7519 62.617,-53.5918 61.5329,-54.4082 60.4024,-55.1988 59.2299,-55.9615 58.0197,-56.6941 56.7755,-57.3946 55.5012,-58.0612 54.2002,-58.692 52.8757,-59.2853 51.5309,-59.8394 50.1684,-60.3528 48.7908,-60.8241 47.4003,-61.2521 45.9989,-61.6355 44.5886,-61.9734 43.1708,-62.2647 41.7472,-62.5087 40.3189,-62.7047 38.8872,-62.8521 37.4531,-62.9507 36.0175,-63 34.5815,-63 33.146,-62.9507 31.7119,-62.8521 30.2801,-62.7047 28.8519,-62.5087 27.4282,-62.2647 26.0105,-61.9734 24.6001,-61.6355 23.1988,-61.2521 21.8083,-60.8241 20.4306,-60.3528 19.0681,-59.8394 17.7233,-59.2853 16.3989,-58.692 15.0979,-58.0612 13.8236,-57.3946 12.5794,-56.6941 11.3691,-55.9615 10.1967,-55.1988 9.0662,-54.4082 7.982,-53.5918 6.9486,-52.7519 5.9706,-51.8907 5.0526,-51.0106 4.1995,-50.114 3.4159,-49.2035 2.7067,-48.2814 2.0765,-47.3503 1.53,-46.4127 1.0715,-45.4713 .7053,-44.5287 .4355,-43.5873 .2657,-42.6497 .1993,-41.7186 .2395,-40.7965 .3888,-39.886 .6495,-38.9894 1.0232,-38.1093 1.5112,-37.2481 2.1141,-36.4082 2.8321,-35.5918 3.6647,-34.8012 4.6109,-34.0385 5.6689,-33.3059 6.8365,-32.6054 8.1108,-31.9388 9.4884,-31.308 10.9651,-30.7147 12.5362,-30.1606 14.1966,-29.6472 15.9404,-29.1759 17.7614,-28.7479 19.6528,-28.3645 21.6074,-28.0266 23.6176,-27.7353 25.6755,-27.4913 27.7728,-27.2953 29.901,-27.1479 32.0515,-27.0493 34.2154,-27 36.3837,-27 38.5476,-27.0493\"/>\n",
       "<text text-anchor=\"middle\" x=\"35.2995\" y=\"-41.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">start</text>\n",
       "</g>\n",
       "<!-- MyMap -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>MyMap</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"152.745\" cy=\"-45\" rx=\"46.2923\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"152.745\" y=\"-41.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">MyMap</text>\n",
       "</g>\n",
       "<!-- _start&#45;&gt;MyMap -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>_start&#45;&gt;MyMap</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M69.9018,-45C78.1609,-45 87.1905,-45 96.1538,-45\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"96.2009,-48.5001 106.2009,-45 96.2009,-41.5001 96.2009,-48.5001\"/>\n",
       "</g>\n",
       "<!-- storey.Extend -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>storey.Extend</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"310.2842\" cy=\"-45\" rx=\"75.2868\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"310.2842\" y=\"-41.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">storey.Extend</text>\n",
       "</g>\n",
       "<!-- MyMap&#45;&gt;storey.Extend -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>MyMap&#45;&gt;storey.Extend</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M199.1598,-45C207.1838,-45 215.7499,-45 224.4316,-45\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"224.6509,-48.5001 234.6509,-45 224.6508,-41.5001 224.6509,-48.5001\"/>\n",
       "</g>\n",
       "<!-- filter -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>filter</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"455.4745\" cy=\"-45\" rx=\"33.5952\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"455.4745\" y=\"-41.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">filter</text>\n",
       "</g>\n",
       "<!-- storey.Extend&#45;&gt;filter -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>storey.Extend&#45;&gt;filter</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M385.8561,-45C394.6736,-45 403.4325,-45 411.6112,-45\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"411.6214,-48.5001 421.6213,-45 411.6213,-41.5001 411.6214,-48.5001\"/>\n",
       "</g>\n",
       "<!-- FeaturesetValidator -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>FeaturesetValidator</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"627.9624\" cy=\"-45\" rx=\"102.8821\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"627.9624\" y=\"-41.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">FeaturesetValidator</text>\n",
       "</g>\n",
       "<!-- filter&#45;&gt;FeaturesetValidator -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>filter&#45;&gt;FeaturesetValidator</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M489.5525,-45C497.3395,-45 506.0352,-45 515.1713,-45\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"515.2357,-48.5001 525.2357,-45 515.2356,-41.5001 515.2357,-48.5001\"/>\n",
       "</g>\n",
       "<!-- Aggregates -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>Aggregates</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"830.9976\" cy=\"-45\" rx=\"64.189\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"830.9976\" y=\"-41.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">Aggregates</text>\n",
       "</g>\n",
       "<!-- FeaturesetValidator&#45;&gt;Aggregates -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>FeaturesetValidator&#45;&gt;Aggregates</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M730.6696,-45C739.3404,-45 747.9997,-45 756.3944,-45\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"756.4857,-48.5001 766.4857,-45 756.4857,-41.5001 756.4857,-48.5001\"/>\n",
       "</g>\n",
       "<!-- parquet -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>parquet</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1004.3418,-86.7273C1004.3418,-88.5331 987.982,-90 967.8418,-90 947.7017,-90 931.3418,-88.5331 931.3418,-86.7273 931.3418,-86.7273 931.3418,-57.2727 931.3418,-57.2727 931.3418,-55.4669 947.7017,-54 967.8418,-54 987.982,-54 1004.3418,-55.4669 1004.3418,-57.2727 1004.3418,-57.2727 1004.3418,-86.7273 1004.3418,-86.7273\"/>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1004.3418,-86.7273C1004.3418,-84.9214 987.982,-83.4545 967.8418,-83.4545 947.7017,-83.4545 931.3418,-84.9214 931.3418,-86.7273\"/>\n",
       "<text text-anchor=\"middle\" x=\"967.8418\" y=\"-68.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">parquet</text>\n",
       "</g>\n",
       "<!-- Aggregates&#45;&gt;parquet -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>Aggregates&#45;&gt;parquet</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M883.9051,-55.4389C896.2444,-57.8735 909.2943,-60.4483 921.3324,-62.8235\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"920.6803,-66.2622 931.1687,-64.7642 922.0353,-59.3946 920.6803,-66.2622\"/>\n",
       "</g>\n",
       "<!-- nosql -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>nosql</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M995.3418,-32.7273C995.3418,-34.5331 983.0159,-36 967.8418,-36 952.6678,-36 940.3418,-34.5331 940.3418,-32.7273 940.3418,-32.7273 940.3418,-3.2727 940.3418,-3.2727 940.3418,-1.4669 952.6678,0 967.8418,0 983.0159,0 995.3418,-1.4669 995.3418,-3.2727 995.3418,-3.2727 995.3418,-32.7273 995.3418,-32.7273\"/>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M995.3418,-32.7273C995.3418,-30.9214 983.0159,-29.4545 967.8418,-29.4545 952.6678,-29.4545 940.3418,-30.9214 940.3418,-32.7273\"/>\n",
       "<text text-anchor=\"middle\" x=\"967.8418\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">nosql</text>\n",
       "</g>\n",
       "<!-- Aggregates&#45;&gt;nosql -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>Aggregates&#45;&gt;nosql</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M883.9051,-34.5611C899.3986,-31.5042 916.0123,-28.2262 930.3369,-25.3999\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"931.1806,-28.801 940.314,-23.4314 929.8256,-21.9334 931.1806,-28.801\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7fa9a4154250>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "quotes_set.graph.to(\"MyMap\", multiplier=3).to(\n",
    "    \"storey.Extend\", _fn=\"({'extra': event['bid'] * 77})\"\n",
    ").to(\"storey.Filter\", \"filter\", _fn=\"(event['bid'] > 51.92)\").to(FeaturesetValidator())\n",
    "\n",
    "quotes_set.add_aggregation(\"ask\", [\"sum\", \"max\"], \"1h\", \"10m\", name=\"asks1\")\n",
    "quotes_set.add_aggregation(\"ask\", [\"sum\", \"max\"], \"5h\", \"10m\", name=\"asks5\")\n",
    "quotes_set.add_aggregation(\"bid\", [\"min\", \"max\"], \"1h\", \"10m\", name=\"bids\")\n",
    "\n",
    "# add feature validation policy\n",
    "quotes_set[\"bid\"] = fstore.Feature(validator=MinMaxValidator(min=52, severity=\"info\"))\n",
    "\n",
    "# add default target definitions and plot\n",
    "quotes_set.set_targets()\n",
    "quotes_set.plot(rankdir=\"LR\", with_targets=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Test and show the pipeline results locally (allow to quickly develop and debug)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "info! bid value is smaller than min, key=['MSFT'] time=2021-05-23 09:04:07.013574 args={'min': 52, 'value': 51.95}\n",
      "info! bid value is smaller than min, key=['MSFT'] time=2021-05-23 09:04:07.020574 args={'min': 52, 'value': 51.97}\n",
      "info! bid value is smaller than min, key=['MSFT'] time=2021-05-23 09:04:07.031574 args={'min': 52, 'value': 51.99}\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>asks1_sum_1h</th>\n",
       "      <th>asks1_max_1h</th>\n",
       "      <th>asks5_sum_5h</th>\n",
       "      <th>asks5_max_5h</th>\n",
       "      <th>bids_min_1h</th>\n",
       "      <th>bids_max_1h</th>\n",
       "      <th>time</th>\n",
       "      <th>bid</th>\n",
       "      <th>ask</th>\n",
       "      <th>multi</th>\n",
       "      <th>extra</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ticker</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>GOOG</th>\n",
       "      <td>720.93</td>\n",
       "      <td>720.93</td>\n",
       "      <td>720.93</td>\n",
       "      <td>720.93</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.50</td>\n",
       "      <td>2021-05-23 09:04:07.013574</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.93</td>\n",
       "      <td>2161.50</td>\n",
       "      <td>55478.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MSFT</th>\n",
       "      <td>51.96</td>\n",
       "      <td>51.96</td>\n",
       "      <td>51.96</td>\n",
       "      <td>51.96</td>\n",
       "      <td>51.95</td>\n",
       "      <td>51.95</td>\n",
       "      <td>2021-05-23 09:04:07.013574</td>\n",
       "      <td>51.95</td>\n",
       "      <td>51.96</td>\n",
       "      <td>155.85</td>\n",
       "      <td>4000.15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MSFT</th>\n",
       "      <td>103.94</td>\n",
       "      <td>51.98</td>\n",
       "      <td>103.94</td>\n",
       "      <td>51.98</td>\n",
       "      <td>51.95</td>\n",
       "      <td>51.97</td>\n",
       "      <td>2021-05-23 09:04:07.020574</td>\n",
       "      <td>51.97</td>\n",
       "      <td>51.98</td>\n",
       "      <td>155.91</td>\n",
       "      <td>4001.69</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MSFT</th>\n",
       "      <td>155.94</td>\n",
       "      <td>52.00</td>\n",
       "      <td>155.94</td>\n",
       "      <td>52.00</td>\n",
       "      <td>51.95</td>\n",
       "      <td>51.99</td>\n",
       "      <td>2021-05-23 09:04:07.031574</td>\n",
       "      <td>51.99</td>\n",
       "      <td>52.00</td>\n",
       "      <td>155.97</td>\n",
       "      <td>4003.23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GOOG</th>\n",
       "      <td>1441.86</td>\n",
       "      <td>720.93</td>\n",
       "      <td>1441.86</td>\n",
       "      <td>720.93</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.50</td>\n",
       "      <td>2021-05-23 09:04:07.038574</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.93</td>\n",
       "      <td>2161.50</td>\n",
       "      <td>55478.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AAPL</th>\n",
       "      <td>98.01</td>\n",
       "      <td>98.01</td>\n",
       "      <td>98.01</td>\n",
       "      <td>98.01</td>\n",
       "      <td>97.99</td>\n",
       "      <td>97.99</td>\n",
       "      <td>2021-05-23 09:04:07.039574</td>\n",
       "      <td>97.99</td>\n",
       "      <td>98.01</td>\n",
       "      <td>293.97</td>\n",
       "      <td>7545.23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GOOG</th>\n",
       "      <td>2162.74</td>\n",
       "      <td>720.93</td>\n",
       "      <td>2162.74</td>\n",
       "      <td>720.93</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.50</td>\n",
       "      <td>2021-05-23 09:04:07.062574</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.88</td>\n",
       "      <td>2161.50</td>\n",
       "      <td>55478.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MSFT</th>\n",
       "      <td>207.97</td>\n",
       "      <td>52.03</td>\n",
       "      <td>207.97</td>\n",
       "      <td>52.03</td>\n",
       "      <td>51.95</td>\n",
       "      <td>52.01</td>\n",
       "      <td>2021-05-23 09:04:07.065574</td>\n",
       "      <td>52.01</td>\n",
       "      <td>52.03</td>\n",
       "      <td>156.03</td>\n",
       "      <td>4004.77</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        asks1_sum_1h  asks1_max_1h  asks5_sum_5h  asks5_max_5h  bids_min_1h  \\\n",
       "ticker                                                                        \n",
       "GOOG          720.93        720.93        720.93        720.93       720.50   \n",
       "MSFT           51.96         51.96         51.96         51.96        51.95   \n",
       "MSFT          103.94         51.98        103.94         51.98        51.95   \n",
       "MSFT          155.94         52.00        155.94         52.00        51.95   \n",
       "GOOG         1441.86        720.93       1441.86        720.93       720.50   \n",
       "AAPL           98.01         98.01         98.01         98.01        97.99   \n",
       "GOOG         2162.74        720.93       2162.74        720.93       720.50   \n",
       "MSFT          207.97         52.03        207.97         52.03        51.95   \n",
       "\n",
       "        bids_max_1h                       time     bid     ask    multi  \\\n",
       "ticker                                                                    \n",
       "GOOG         720.50 2021-05-23 09:04:07.013574  720.50  720.93  2161.50   \n",
       "MSFT          51.95 2021-05-23 09:04:07.013574   51.95   51.96   155.85   \n",
       "MSFT          51.97 2021-05-23 09:04:07.020574   51.97   51.98   155.91   \n",
       "MSFT          51.99 2021-05-23 09:04:07.031574   51.99   52.00   155.97   \n",
       "GOOG         720.50 2021-05-23 09:04:07.038574  720.50  720.93  2161.50   \n",
       "AAPL          97.99 2021-05-23 09:04:07.039574   97.99   98.01   293.97   \n",
       "GOOG         720.50 2021-05-23 09:04:07.062574  720.50  720.88  2161.50   \n",
       "MSFT          52.01 2021-05-23 09:04:07.065574   52.01   52.03   156.03   \n",
       "\n",
       "           extra  \n",
       "ticker            \n",
       "GOOG    55478.50  \n",
       "MSFT     4000.15  \n",
       "MSFT     4001.69  \n",
       "MSFT     4003.23  \n",
       "GOOG    55478.50  \n",
       "AAPL     7545.23  \n",
       "GOOG    55478.50  \n",
       "MSFT     4004.77  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fstore.preview(\n",
    "    quotes_set,\n",
    "    quotes,\n",
    "    entity_columns=[\"ticker\"],\n",
    "    timestamp_key=\"time\",\n",
    "    options=fstore.InferOptions.default(),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "tags": [
     "hide-output"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "kind: FeatureSet\n",
      "metadata:\n",
      "  name: stock-quotes\n",
      "spec:\n",
      "  entities:\n",
      "  - name: ticker\n",
      "    value_type: str\n",
      "  features:\n",
      "  - name: asks1_sum_1h\n",
      "    value_type: float\n",
      "    aggregate: true\n",
      "  - name: asks1_max_1h\n",
      "    value_type: float\n",
      "    aggregate: true\n",
      "  - name: asks5_sum_5h\n",
      "    value_type: float\n",
      "    aggregate: true\n",
      "  - name: asks5_max_5h\n",
      "    value_type: float\n",
      "    aggregate: true\n",
      "  - name: bids_min_1h\n",
      "    value_type: float\n",
      "    aggregate: true\n",
      "  - name: bids_max_1h\n",
      "    value_type: float\n",
      "    aggregate: true\n",
      "  - name: bid\n",
      "    value_type: float\n",
      "    validator:\n",
      "      kind: minmax\n",
      "      severity: info\n",
      "      min: 52\n",
      "  - name: ask\n",
      "    value_type: float\n",
      "  - name: multi\n",
      "    value_type: float\n",
      "  - name: extra\n",
      "    value_type: float\n",
      "  partition_keys: []\n",
      "  timestamp_key: time\n",
      "  source:\n",
      "    path: None\n",
      "  targets:\n",
      "  - name: parquet\n",
      "    kind: parquet\n",
      "  - name: nosql\n",
      "    kind: nosql\n",
      "  graph:\n",
      "    states:\n",
      "      MyMap:\n",
      "        kind: task\n",
      "        class_name: MyMap\n",
      "        class_args:\n",
      "          multiplier: 3\n",
      "      storey.Extend:\n",
      "        kind: task\n",
      "        class_name: storey.Extend\n",
      "        class_args:\n",
      "          _fn: '({''extra'': event[''bid''] * 77})'\n",
      "        after:\n",
      "        - MyMap\n",
      "      filter:\n",
      "        kind: task\n",
      "        class_name: storey.Filter\n",
      "        class_args:\n",
      "          _fn: (event['bid'] > 51.92)\n",
      "        after:\n",
      "        - storey.Extend\n",
      "      FeaturesetValidator:\n",
      "        kind: task\n",
      "        class_name: mlrun.feature_store.steps.FeaturesetValidator\n",
      "        class_args:\n",
      "          featureset: .\n",
      "          columns: null\n",
      "        after:\n",
      "        - filter\n",
      "      Aggregates:\n",
      "        kind: task\n",
      "        class_name: storey.AggregateByKey\n",
      "        class_args:\n",
      "          aggregates:\n",
      "          - name: asks1\n",
      "            column: ask\n",
      "            operations:\n",
      "            - sum\n",
      "            - max\n",
      "            windows:\n",
      "            - 1h\n",
      "            period: 10m\n",
      "          - name: asks5\n",
      "            column: ask\n",
      "            operations:\n",
      "            - sum\n",
      "            - max\n",
      "            windows:\n",
      "            - 5h\n",
      "            period: 10m\n",
      "          - name: bids\n",
      "            column: bid\n",
      "            operations:\n",
      "            - min\n",
      "            - max\n",
      "            windows:\n",
      "            - 1h\n",
      "            period: 10m\n",
      "          table: .\n",
      "        after:\n",
      "        - FeaturesetValidator\n",
      "  output_path: v3io:///projects/{{run.project}}/artifacts\n",
      "status:\n",
      "  state: created\n",
      "  stats:\n",
      "    ticker:\n",
      "      count: 8\n",
      "      unique: 3\n",
      "      top: MSFT\n",
      "      freq: 4\n",
      "    asks1_sum_1h:\n",
      "      count: 8.0\n",
      "      mean: 617.9187499999999\n",
      "      min: 51.96\n",
      "      max: 2162.74\n",
      "      std: 784.8779804245735\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 1\n",
      "      - - 51.96\n",
      "        - 157.499\n",
      "        - 263.03799999999995\n",
      "        - 368.57699999999994\n",
      "        - 474.11599999999993\n",
      "        - 579.655\n",
      "        - 685.194\n",
      "        - 790.733\n",
      "        - 896.2719999999999\n",
      "        - 1001.8109999999999\n",
      "        - 1107.35\n",
      "        - 1212.889\n",
      "        - 1318.4279999999999\n",
      "        - 1423.9669999999999\n",
      "        - 1529.5059999999999\n",
      "        - 1635.0449999999998\n",
      "        - 1740.5839999999998\n",
      "        - 1846.1229999999998\n",
      "        - 1951.6619999999998\n",
      "        - 2057.2009999999996\n",
      "        - 2162.74\n",
      "    asks1_max_1h:\n",
      "      count: 8.0\n",
      "      mean: 308.59625\n",
      "      min: 51.96\n",
      "      max: 720.93\n",
      "      std: 341.7989955655851\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 3\n",
      "      - - 51.96\n",
      "        - 85.4085\n",
      "        - 118.857\n",
      "        - 152.3055\n",
      "        - 185.754\n",
      "        - 219.2025\n",
      "        - 252.65099999999998\n",
      "        - 286.0995\n",
      "        - 319.54799999999994\n",
      "        - 352.9964999999999\n",
      "        - 386.44499999999994\n",
      "        - 419.89349999999996\n",
      "        - 453.3419999999999\n",
      "        - 486.7904999999999\n",
      "        - 520.2389999999999\n",
      "        - 553.6875\n",
      "        - 587.136\n",
      "        - 620.5844999999999\n",
      "        - 654.0329999999999\n",
      "        - 687.4815\n",
      "        - 720.93\n",
      "    asks5_sum_5h:\n",
      "      count: 8.0\n",
      "      mean: 617.9187499999999\n",
      "      min: 51.96\n",
      "      max: 2162.74\n",
      "      std: 784.8779804245735\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 1\n",
      "      - - 51.96\n",
      "        - 157.499\n",
      "        - 263.03799999999995\n",
      "        - 368.57699999999994\n",
      "        - 474.11599999999993\n",
      "        - 579.655\n",
      "        - 685.194\n",
      "        - 790.733\n",
      "        - 896.2719999999999\n",
      "        - 1001.8109999999999\n",
      "        - 1107.35\n",
      "        - 1212.889\n",
      "        - 1318.4279999999999\n",
      "        - 1423.9669999999999\n",
      "        - 1529.5059999999999\n",
      "        - 1635.0449999999998\n",
      "        - 1740.5839999999998\n",
      "        - 1846.1229999999998\n",
      "        - 1951.6619999999998\n",
      "        - 2057.2009999999996\n",
      "        - 2162.74\n",
      "    asks5_max_5h:\n",
      "      count: 8.0\n",
      "      mean: 308.59625\n",
      "      min: 51.96\n",
      "      max: 720.93\n",
      "      std: 341.7989955655851\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 3\n",
      "      - - 51.96\n",
      "        - 85.4085\n",
      "        - 118.857\n",
      "        - 152.3055\n",
      "        - 185.754\n",
      "        - 219.2025\n",
      "        - 252.65099999999998\n",
      "        - 286.0995\n",
      "        - 319.54799999999994\n",
      "        - 352.9964999999999\n",
      "        - 386.44499999999994\n",
      "        - 419.89349999999996\n",
      "        - 453.3419999999999\n",
      "        - 486.7904999999999\n",
      "        - 520.2389999999999\n",
      "        - 553.6875\n",
      "        - 587.136\n",
      "        - 620.5844999999999\n",
      "        - 654.0329999999999\n",
      "        - 687.4815\n",
      "        - 720.93\n",
      "    bids_min_1h:\n",
      "      count: 8.0\n",
      "      mean: 308.41125\n",
      "      min: 51.95\n",
      "      max: 720.5\n",
      "      std: 341.59667259325835\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 3\n",
      "      - - 51.95\n",
      "        - 85.3775\n",
      "        - 118.80499999999999\n",
      "        - 152.2325\n",
      "        - 185.65999999999997\n",
      "        - 219.08749999999998\n",
      "        - 252.515\n",
      "        - 285.94249999999994\n",
      "        - 319.36999999999995\n",
      "        - 352.79749999999996\n",
      "        - 386.22499999999997\n",
      "        - 419.6524999999999\n",
      "        - 453.0799999999999\n",
      "        - 486.50749999999994\n",
      "        - 519.935\n",
      "        - 553.3625\n",
      "        - 586.79\n",
      "        - 620.2175\n",
      "        - 653.645\n",
      "        - 687.0725\n",
      "        - 720.5\n",
      "    bids_max_1h:\n",
      "      count: 8.0\n",
      "      mean: 308.42625\n",
      "      min: 51.95\n",
      "      max: 720.5\n",
      "      std: 341.58380276661245\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 3\n",
      "      - - 51.95\n",
      "        - 85.3775\n",
      "        - 118.80499999999999\n",
      "        - 152.2325\n",
      "        - 185.65999999999997\n",
      "        - 219.08749999999998\n",
      "        - 252.515\n",
      "        - 285.94249999999994\n",
      "        - 319.36999999999995\n",
      "        - 352.79749999999996\n",
      "        - 386.22499999999997\n",
      "        - 419.6524999999999\n",
      "        - 453.0799999999999\n",
      "        - 486.50749999999994\n",
      "        - 519.935\n",
      "        - 553.3625\n",
      "        - 586.79\n",
      "        - 620.2175\n",
      "        - 653.645\n",
      "        - 687.0725\n",
      "        - 720.5\n",
      "    time:\n",
      "      count: 8\n",
      "      mean: '2021-05-23 09:04:07.035699200'\n",
      "      min: '2021-05-23 09:04:07.013574'\n",
      "      max: '2021-05-23 09:04:07.065574'\n",
      "    bid:\n",
      "      count: 8.0\n",
      "      mean: 308.42625\n",
      "      min: 51.95\n",
      "      max: 720.5\n",
      "      std: 341.58380276661245\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 3\n",
      "      - - 51.95\n",
      "        - 85.3775\n",
      "        - 118.80499999999999\n",
      "        - 152.2325\n",
      "        - 185.65999999999997\n",
      "        - 219.08749999999998\n",
      "        - 252.515\n",
      "        - 285.94249999999994\n",
      "        - 319.36999999999995\n",
      "        - 352.79749999999996\n",
      "        - 386.22499999999997\n",
      "        - 419.6524999999999\n",
      "        - 453.0799999999999\n",
      "        - 486.50749999999994\n",
      "        - 519.935\n",
      "        - 553.3625\n",
      "        - 586.79\n",
      "        - 620.2175\n",
      "        - 653.645\n",
      "        - 687.0725\n",
      "        - 720.5\n",
      "    ask:\n",
      "      count: 8.0\n",
      "      mean: 308.59\n",
      "      min: 51.96\n",
      "      max: 720.93\n",
      "      std: 341.79037903369954\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 3\n",
      "      - - 51.96\n",
      "        - 85.4085\n",
      "        - 118.857\n",
      "        - 152.3055\n",
      "        - 185.754\n",
      "        - 219.2025\n",
      "        - 252.65099999999998\n",
      "        - 286.0995\n",
      "        - 319.54799999999994\n",
      "        - 352.9964999999999\n",
      "        - 386.44499999999994\n",
      "        - 419.89349999999996\n",
      "        - 453.3419999999999\n",
      "        - 486.7904999999999\n",
      "        - 520.2389999999999\n",
      "        - 553.6875\n",
      "        - 587.136\n",
      "        - 620.5844999999999\n",
      "        - 654.0329999999999\n",
      "        - 687.4815\n",
      "        - 720.93\n",
      "    multi:\n",
      "      count: 8.0\n",
      "      mean: 925.27875\n",
      "      min: 155.85000000000002\n",
      "      max: 2161.5\n",
      "      std: 1024.7514082998375\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 3\n",
      "      - - 155.85000000000002\n",
      "        - 256.13250000000005\n",
      "        - 356.415\n",
      "        - 456.6975\n",
      "        - 556.98\n",
      "        - 657.2625\n",
      "        - 757.545\n",
      "        - 857.8275\n",
      "        - 958.11\n",
      "        - 1058.3925\n",
      "        - 1158.6750000000002\n",
      "        - 1258.9575\n",
      "        - 1359.2399999999998\n",
      "        - 1459.5225\n",
      "        - 1559.8049999999998\n",
      "        - 1660.0875\n",
      "        - 1760.37\n",
      "        - 1860.6525000000001\n",
      "        - 1960.935\n",
      "        - 2061.2175\n",
      "        - 2161.5\n",
      "    extra:\n",
      "      count: 8.0\n",
      "      mean: 23748.82125\n",
      "      min: 4000.15\n",
      "      max: 55478.5\n",
      "      std: 26301.95281302916\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 3\n",
      "      - - 4000.15\n",
      "        - 6574.0675\n",
      "        - 9147.985\n",
      "        - 11721.9025\n",
      "        - 14295.82\n",
      "        - 16869.7375\n",
      "        - 19443.655000000002\n",
      "        - 22017.572500000002\n",
      "        - 24591.49\n",
      "        - 27165.4075\n",
      "        - 29739.325\n",
      "        - 32313.2425\n",
      "        - 34887.16\n",
      "        - 37461.0775\n",
      "        - 40034.995\n",
      "        - 42608.9125\n",
      "        - 45182.83\n",
      "        - 47756.747500000005\n",
      "        - 50330.665\n",
      "        - 52904.582500000004\n",
      "        - 55478.5\n",
      "  preview:\n",
      "  - - asks1_sum_1h\n",
      "    - asks1_max_1h\n",
      "    - asks5_sum_5h\n",
      "    - asks5_max_5h\n",
      "    - bids_min_1h\n",
      "    - bids_max_1h\n",
      "    - time\n",
      "    - bid\n",
      "    - ask\n",
      "    - multi\n",
      "    - extra\n",
      "  - - 720.93\n",
      "    - 720.93\n",
      "    - 720.93\n",
      "    - 720.93\n",
      "    - 720.5\n",
      "    - 720.5\n",
      "    - 2021-05-23T09:04:07.013574\n",
      "    - 720.5\n",
      "    - 720.93\n",
      "    - 2161.5\n",
      "    - 55478.5\n",
      "  - - 51.96\n",
      "    - 51.96\n",
      "    - 51.96\n",
      "    - 51.96\n",
      "    - 51.95\n",
      "    - 51.95\n",
      "    - 2021-05-23T09:04:07.013574\n",
      "    - 51.95\n",
      "    - 51.96\n",
      "    - 155.85000000000002\n",
      "    - 4000.15\n",
      "  - - 103.94\n",
      "    - 51.98\n",
      "    - 103.94\n",
      "    - 51.98\n",
      "    - 51.95\n",
      "    - 51.97\n",
      "    - 2021-05-23T09:04:07.020574\n",
      "    - 51.97\n",
      "    - 51.98\n",
      "    - 155.91\n",
      "    - 4001.69\n",
      "  - - 155.94\n",
      "    - 52.0\n",
      "    - 155.94\n",
      "    - 52.0\n",
      "    - 51.95\n",
      "    - 51.99\n",
      "    - 2021-05-23T09:04:07.031574\n",
      "    - 51.99\n",
      "    - 52.0\n",
      "    - 155.97\n",
      "    - 4003.23\n",
      "  - - 1441.86\n",
      "    - 720.93\n",
      "    - 1441.86\n",
      "    - 720.93\n",
      "    - 720.5\n",
      "    - 720.5\n",
      "    - 2021-05-23T09:04:07.038574\n",
      "    - 720.5\n",
      "    - 720.93\n",
      "    - 2161.5\n",
      "    - 55478.5\n",
      "  - - 98.01\n",
      "    - 98.01\n",
      "    - 98.01\n",
      "    - 98.01\n",
      "    - 97.99\n",
      "    - 97.99\n",
      "    - 2021-05-23T09:04:07.039574\n",
      "    - 97.99\n",
      "    - 98.01\n",
      "    - 293.96999999999997\n",
      "    - 7545.23\n",
      "  - - 2162.74\n",
      "    - 720.93\n",
      "    - 2162.74\n",
      "    - 720.93\n",
      "    - 720.5\n",
      "    - 720.5\n",
      "    - 2021-05-23T09:04:07.062574\n",
      "    - 720.5\n",
      "    - 720.88\n",
      "    - 2161.5\n",
      "    - 55478.5\n",
      "  - - 207.97\n",
      "    - 52.03\n",
      "    - 207.97\n",
      "    - 52.03\n",
      "    - 51.95\n",
      "    - 52.01\n",
      "    - 2021-05-23T09:04:07.065574\n",
      "    - 52.01\n",
      "    - 52.03\n",
      "    - 156.03\n",
      "    - 4004.77\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# print the feature set object\n",
    "print(quotes_set.to_yaml())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Ingest data into offline and online stores\n",
    "This writes to both targets (Parquet and NoSQL)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "info! bid value is smaller than min, key=['MSFT'] time=2021-05-23 09:04:07.013574 args={'min': 52, 'value': 51.95}\n",
      "info! bid value is smaller than min, key=['MSFT'] time=2021-05-23 09:04:07.020574 args={'min': 52, 'value': 51.97}\n",
      "info! bid value is smaller than min, key=['MSFT'] time=2021-05-23 09:04:07.031574 args={'min': 52, 'value': 51.99}\n",
      "info! bid value is smaller than min, key=['MSFT'] time=2021-05-23 09:04:07.013574 args={'min': 52, 'value': 51.95}\n",
      "info! bid value is smaller than min, key=['MSFT'] time=2021-05-23 09:04:07.020574 args={'min': 52, 'value': 51.97}\n",
      "info! bid value is smaller than min, key=['MSFT'] time=2021-05-23 09:04:07.031574 args={'min': 52, 'value': 51.99}\n"
     ]
    }
   ],
   "source": [
    "# save ingest data and print the FeatureSet spec\n",
    "df = fstore.ingest(quotes_set, quotes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get an offline feature vector for training\n",
    "Example of combining features from 3 sources with time travel join of 3 tables with **time travel**.\n",
    "\n",
    "Specify a set of features and request the feature vector offline result as a dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "features = [\n",
    "    \"stock-quotes.multi\",\n",
    "    \"stock-quotes.asks5_sum_5h as total_ask\",\n",
    "    \"stock-quotes.bids_min_1h\",\n",
    "    \"stock-quotes.bids_max_1h\",\n",
    "    \"stocks.*\",\n",
    "]\n",
    "\n",
    "vector = fstore.FeatureVector(\n",
    "    \"stocks-vec\", features, description=\"stocks demo feature vector\"\n",
    ")\n",
    "vector.save()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>price</th>\n",
       "      <th>quantity</th>\n",
       "      <th>multi</th>\n",
       "      <th>total_ask</th>\n",
       "      <th>bids_min_1h</th>\n",
       "      <th>bids_max_1h</th>\n",
       "      <th>name</th>\n",
       "      <th>exchange</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>51.95</td>\n",
       "      <td>75</td>\n",
       "      <td>155.97</td>\n",
       "      <td>155.94</td>\n",
       "      <td>51.95</td>\n",
       "      <td>51.99</td>\n",
       "      <td>Microsoft Corporation</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>51.95</td>\n",
       "      <td>155</td>\n",
       "      <td>155.97</td>\n",
       "      <td>155.94</td>\n",
       "      <td>51.95</td>\n",
       "      <td>51.99</td>\n",
       "      <td>Microsoft Corporation</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>720.77</td>\n",
       "      <td>100</td>\n",
       "      <td>2161.50</td>\n",
       "      <td>2162.74</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.50</td>\n",
       "      <td>Alphabet Inc</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>720.92</td>\n",
       "      <td>100</td>\n",
       "      <td>2161.50</td>\n",
       "      <td>2162.74</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.50</td>\n",
       "      <td>Alphabet Inc</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>98.00</td>\n",
       "      <td>100</td>\n",
       "      <td>293.97</td>\n",
       "      <td>98.01</td>\n",
       "      <td>97.99</td>\n",
       "      <td>97.99</td>\n",
       "      <td>Apple Inc</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    price  quantity    multi  total_ask  bids_min_1h  bids_max_1h  \\\n",
       "0   51.95        75   155.97     155.94        51.95        51.99   \n",
       "1   51.95       155   155.97     155.94        51.95        51.99   \n",
       "2  720.77       100  2161.50    2162.74       720.50       720.50   \n",
       "3  720.92       100  2161.50    2162.74       720.50       720.50   \n",
       "4   98.00       100   293.97      98.01        97.99        97.99   \n",
       "\n",
       "                    name exchange  \n",
       "0  Microsoft Corporation   NASDAQ  \n",
       "1  Microsoft Corporation   NASDAQ  \n",
       "2           Alphabet Inc   NASDAQ  \n",
       "3           Alphabet Inc   NASDAQ  \n",
       "4              Apple Inc   NASDAQ  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "resp = fstore.get_offline_features(\n",
    "    vector, entity_rows=trades, entity_timestamp_column=\"time\"\n",
    ")\n",
    "resp.to_dataframe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initialize an online feature service and use it for real-time inference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "service = fstore.get_online_feature_service(\"stocks-vec\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Request feature vector statistics, can be used for imputing or validation**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>min</th>\n",
       "      <th>max</th>\n",
       "      <th>std</th>\n",
       "      <th>hist</th>\n",
       "      <th>unique</th>\n",
       "      <th>top</th>\n",
       "      <th>freq</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>multi</th>\n",
       "      <td>8.0</td>\n",
       "      <td>925.27875</td>\n",
       "      <td>155.85</td>\n",
       "      <td>2161.50</td>\n",
       "      <td>1024.751408</td>\n",
       "      <td>[[4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>total_ask</th>\n",
       "      <td>8.0</td>\n",
       "      <td>617.91875</td>\n",
       "      <td>51.96</td>\n",
       "      <td>2162.74</td>\n",
       "      <td>784.877980</td>\n",
       "      <td>[[4, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0,...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bids_min_1h</th>\n",
       "      <td>8.0</td>\n",
       "      <td>308.41125</td>\n",
       "      <td>51.95</td>\n",
       "      <td>720.50</td>\n",
       "      <td>341.596673</td>\n",
       "      <td>[[4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bids_max_1h</th>\n",
       "      <td>8.0</td>\n",
       "      <td>308.42625</td>\n",
       "      <td>51.95</td>\n",
       "      <td>720.50</td>\n",
       "      <td>341.583803</td>\n",
       "      <td>[[4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Alphabet Inc</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>exchange</th>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NASDAQ</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             count       mean     min      max          std  \\\n",
       "multi          8.0  925.27875  155.85  2161.50  1024.751408   \n",
       "total_ask      8.0  617.91875   51.96  2162.74   784.877980   \n",
       "bids_min_1h    8.0  308.41125   51.95   720.50   341.596673   \n",
       "bids_max_1h    8.0  308.42625   51.95   720.50   341.583803   \n",
       "name           3.0        NaN     NaN      NaN          NaN   \n",
       "exchange       3.0        NaN     NaN      NaN          NaN   \n",
       "\n",
       "                                                          hist  unique  \\\n",
       "multi        [[4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...     NaN   \n",
       "total_ask    [[4, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0,...     NaN   \n",
       "bids_min_1h  [[4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...     NaN   \n",
       "bids_max_1h  [[4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...     NaN   \n",
       "name                                                       NaN     3.0   \n",
       "exchange                                                   NaN     1.0   \n",
       "\n",
       "                      top  freq  \n",
       "multi                 NaN   NaN  \n",
       "total_ask             NaN   NaN  \n",
       "bids_min_1h           NaN   NaN  \n",
       "bids_max_1h           NaN   NaN  \n",
       "name         Alphabet Inc   1.0  \n",
       "exchange           NASDAQ   3.0  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "service.vector.get_stats_table()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Real-time feature vector request**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'asks5_sum_5h': 2162.74,\n",
       "  'bids_min_1h': 720.5,\n",
       "  'bids_max_1h': 720.5,\n",
       "  'multi': 2161.5,\n",
       "  'name': 'Alphabet Inc',\n",
       "  'exchange': 'NASDAQ',\n",
       "  'total_ask': None},\n",
       " {'asks5_sum_5h': 207.97,\n",
       "  'bids_min_1h': 51.95,\n",
       "  'bids_max_1h': 52.01,\n",
       "  'multi': 156.03,\n",
       "  'name': 'Microsoft Corporation',\n",
       "  'exchange': 'NASDAQ',\n",
       "  'total_ask': None}]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "service.get([{\"ticker\": \"GOOG\"}, {\"ticker\": \"MSFT\"}])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'asks5_sum_5h': 98.01,\n",
       "  'bids_min_1h': 97.99,\n",
       "  'bids_max_1h': 97.99,\n",
       "  'multi': 293.97,\n",
       "  'name': 'Apple Inc',\n",
       "  'exchange': 'NASDAQ',\n",
       "  'total_ask': None}]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "service.get([{\"ticker\": \"AAPL\"}])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "service.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
